I2VM: Incremental import vector machines

a r t i c l e i n f o We introduce an innovative incremental learner called incremental import vector machines (I 2 VM). The kernel-based discriminative approach is able to deal with complex data distributions. Additionally, the learner is sparse for an efficient training and testing and has a probabilistic output. We particularly investigate the reconstructive component of import vector machines, in order to use it for robust incremental learning. By performing incre-mental update steps, we are able to add and remove data samples, as well as update the current set of model parameters for incremental learning. By using various standard benchmarks, we demonstrate how I 2 VM is competitive or superior to other incremental methods. It is also shown that our approach is capable of managing concept-drifts in the data distributions. Incremental learning methods have obtained large interest in areas in which a sequential data treatment is preferred, e.g. in tracking applications or time series analysis. Compared to batch learning methods in which data is available and simultaneously processable, incremental methods have to deal with sequentially incoming data, of which order and number can be arbitrary. A powerful incremental learning should have the following requirements: (a) It should perform comparably to its batch learning counterpart. When applied to the same data samples, it should be independent from the sequence of the data. Thus, it should be able to handle real or pseudo-concept-drifts in data distribution without suffering a loss in performance. (b) It should separate the classes well, regardless of the complexity of data distribution. (c) It should be able to deal with arbitrarily long data streams, e.g. in order to treat lifelong learning tasks. (d) It should provide reliable posterior probabilities in order to allow for a meaningful evaluation, to serve as input for further processing steps, as e.g. graphical models, or to provide criteria to decide on irrelevant data to keep the model sparse. To meet requirement (a) and (b), incremental methods in general need a reconstructive and should have a discriminative model component. A reconstructive model component represents the significant sub‐domain of the data distribution. It offers robustness against the sequence of data samples, as well as being effective in the case of many competing classes. Additionally, it is also capable of adapting to actual or apparent changes in the data distribution, which appear as concept-drifts to the learner, caused by either changes within …